Stereoscopic Image Display Apparatus, Stereoscopic Image Displaying Method And Computer Program Product

ABSTRACT

A stereoscopic image display apparatus which can accurately visually recognize all the regions of a stereoscopic image without using a varifocal lens, and can form a natural three-dimensional image on a retina with a processing load on a computer eased even if an image is viewed by a plurality of viewers from any positions. A stereoscopic image display apparatus for generating a stereoscopic image that forms three-dimensional image on a retina of a viewer and displaying it, wherein a critical parallax that is the boundary of parallax capable of forming a three-dimensional image on a retina of a viewer is calculated, the dimensions of rectangular parallelepiped inscribing a sphere having a diameter as the calculated critical parallax are calculated, a space including an object is divided into a plurality of spaces using the calculated rectangular parallelepiped, a stereoscopic image of the object with respect to a single gazing point is generated for each divided space, and the plurality of generated stereoscopic images are pasted together to generate a single stereoscopic image and display the generated single stereoscopic image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. national phase application under 35 U.S.C.§371 of International Patent Application No. PCT/JP2005/022762, filed onDec. 12, 2005, and claims the benefit of Japanese Patent Application No.2005-080482, filed on Mar. 18, 2005, both of which are incorporated byreference herein. The International Application was published inJapanese on Sep. 28, 2006 as International Publication No. WO2006/100805 A1 under PCT Article 21(2).

TECHNICAL FIELD

The present invention relates to a stereoscopic image display apparatus,a stereoscopic image displaying method and a computer program productfor generating and displaying a stereoscopic image, capable of forming athree-dimensional image on a retina regardless of the viewing positionof a viewer and capable of allowing the viewer to naturally grasp thespatial positional relationship of an object by using the parallaxbetween the eyes of the viewer.

BACKGROUND ART

With the rapid progress of image processing technology in recent years,not only stereoscopic image display apparatuses that generatestereoscopic images viewers can enjoy using special glasses havingoptical lenses, such as polarized lenses, but also stereoscopic imagedisplay apparatuses that generate stereoscopic images viewers can enjoywith the naked eyes are being developed in large quantities. In suchconventional stereoscopic image display apparatuses, a stereoscopicimage with respect to a single gazing point is generated and displayed.Since some regions of a space can be recognized clearly, the spatialpositional relationship can be recognized accurately. However, otherregions are recognized as the so-called blurred images, and there occursa problem of being difficult to accurately recognize the positionalrelationship in the entire region of the space.

FIG. 16 is a schematic view showing the concept of a conventionalstereoscopic image display system. When a viewer sees the gazing point 2of a projected object (a vehicle in FIG. 16) via a screen 1 to which astereoscopic image is projected, the stereoscopic image can berecognized clearly since no parallax occurs at the gazing point 2. Onthe other hand, in the case of a stereoscopic image away from the gazingpoint 2, as the parallax is larger, the divergence between thethree-dimensional image 3 formed on the left retina and thethree-dimensional image 4 formed on the right retina becomes larger, andthe stereoscopic image is eventually recognized in a blurred state. FIG.17 is a view showing regions in which image blurring occurs in theconventional stereoscopic image display system. As shown in FIG. 17, inthe conventional stereoscopic image display system, fixed focal lengthlenses 5, 5, . . . are used, and the object near the gazing point 2 canbe recognized clearly. However, in regions away from the gazing point 2,for example, in hatched regions 6, the parallax between the left andright images is larger than a predetermined value, and images arerecognized as blurred images.

For the purpose of solving these problems, for example, in JapanesePatent No. 3064992 and Japanese Patent Application Laid-open No.2001-238229, gazing points are provided at multiple different positionsbetween the viewer and the screen 1 using varifocal lenses, and theregions in which images can be visually recognized clearly are extended,whereby differences in the way how a stereoscopic image is seendepending on the differences in the position of the viewer, the gazingpoint, etc. are equalized. FIG. 18 is a view showing regions in whichimage blurring occurs in a stereoscopic image display systemincorporating varifocal lenses. As shown in FIG. 18, multiple gazingpoints 2, 2, . . . , can be provided by using the varifocal lenses 7, 7,. . . . Furthermore, the hatched regions 6 in FIG. 17, that is, theregions in which the parallax between the right and left images islarger than the predetermined value and images are recognized as blurredimages, can be reduced or eliminated by using the multiple varifocallenses 7, 7, . . . and by ingeniously disposing the gazing points 2, 2,. . .

SUMMARY

However, for the purpose of completely eliminating the regions in whichimages are blurred in the stereoscopic image display systemincorporating the above-mentioned varifocal lenses 7, 7, . . . , it isnecessary to provide the varifocal lenses 7 in large quantities, andthere is a problem of increasing cost.

Furthermore, even if the varifocal lenses 7 are provided in largequantities, for example, in the case of an image of an object beingpresent at a position away from any gazing points, the parallax betweenthe right and left images becomes larger than the predetermined value atany gazing points, and the image is recognized as a blurred image. Forthis reason, even when the viewer frequently changes his/her viewpointto grasp the spatial positional relationship of the object, the imagescan only be recognized as blurred images, and there is a problem ofbeing unable to completely eliminate regions wherein images arerecognized as blurred images.

This problem is caused by the fact that the limit value of the parallax(hereafter referred to as critical parallax) within which the viewer canrecognize a stereoscopic image without blurring is not considered in theconventional stereoscopic image display system.

The critical parallax can be obtained on the basis of the positionalrelationship between the viewer and the screen.

In addition, in the conventional stereoscopic image display system,comparison judgment processing must be carried out for all the pixelsincluded in an image so that the image being most sharply focused isextracted from multiple photographed images. In other words, the degreesof correlation are calculated for all the pixels, and the parallaxes arecalculated. Then, the calculated parallaxes must be compared amongmultiple images, whereby there causes a problem that the load forcomputer processing becomes enormous.

Accordingly, an object of the present invention is to provide astereoscopic image display apparatus, a stereoscopic image displayingmethod and a computer program product capable of allowing a viewer tovisually recognize all the regions of a stereoscopic image accuratelywithout using varifocal lenses and capable of reducing the load forcomputer processing.

Another object of the present invention is to provide a stereoscopicimage display apparatus, a stereoscopic image displaying method and acomputer program product capable of calculating a critical parallax asaccurately as possible and forming a natural three-dimensional image ona retina even if a stereoscopic image is viewed by multiple viewers fromany positions.

In order to attain the above-mentioned objects, a stereoscopic imagedisplay apparatus according to a first aspect is characterized by astereoscopic image display apparatus for generating and displaying astereoscopic image that forms a three-dimensional image on a retina of aviewer, characterized by comprising:

critical parallax calculating means for calculating a critical parallaxthat is the boundary of a parallax capable of forming athree-dimensional image on a retina of the viewer;

rectangular parallelepiped dimension calculating means for calculatingthe dimensions of a rectangular parallelepiped inscribing a spherehaving a diameter equal to the calculated critical parallax;

space dividing means for dividing a space including an object intomultiple spaces using the calculated rectangular parallelepiped;

gazing point image generating means for generating a stereoscopic imageof the object with respect to a single gazing point for each dividedspace;

gazing point image pasting means for pasting the generated multiplestereoscopic images together to generate a single stereoscopic image;and

image displaying means for displaying the generated single stereoscopicimage.

In addition, a stereoscopic image display apparatus according to asecond aspect is characterized by the stereoscopic image displayapparatus, characterized by comprising relative position detecting meansfor detecting the relative position between the screen on which astereoscopic image is displayed and the viewer,

wherein said critical parallax is calculated as a parallax in which thedifference between the inverse of the distance from the point at whichthe optical axes of the right and left eyes intersect to the eyes andthe inverse of the distance from the screen on which the stereoscopicimage is displayed to the eyes has a predetermined value, in the firstaspect.

Furthermore, a stereoscopic image display apparatus according to a thirdaspect is characterized by the stereoscopic image display apparatus,characterized in that said critical parallax calculating means isequipped with correcting means for correcting said critical parallax onthe basis of the personal characteristics of the viewer, in the first orsecond aspect.

Moreover, a stereoscopic image display apparatus according to a fourthaspect is characterized by the stereoscopic image display apparatus,characterized in that

said gazing point image pasting means comprises:

means for calculating the brightness difference, that is, by using onespace of the divided spaces as a reference, the difference between thebrightness value of a stereoscopic image generated in said one space andthe brightness value of a stereoscopic image generated in another spaceadjacent thereto;

means for moving the other space in parallel with said one space and forrecalculating the brightness difference; and

means for obtaining the relative position of the other space withrespect to said one space, at which the calculated brightness differencebecomes minimal, in any one of the first through third aspects.

Still further, a stereoscopic image displaying method according to afifth aspect is characterized by a stereoscopic image displaying methodfor generating and displaying a stereoscopic image that forms athree-dimensional image on a retina of a viewer, characterized bycomprising:

calculating a critical parallax that is the boundary of a parallaxcapable of forming a three-dimensional image on a retina of the viewer;

calculating the dimensions of a rectangular parallelepiped inscribing asphere having a diameter equal to the calculated critical parallax;

dividing a space including an object into multiple spaces using thecalculated rectangular parallelepiped;

generating a stereoscopic image of the object with respect to a singlegazing point for each divided space;

pasting the generated multiple stereoscopic images together to generatea single stereoscopic image; and

displaying the generated single stereoscopic image.

Still further, a stereoscopic image displaying method according to asixth aspect is characterized by the stereoscopic image displayingmethod, characterized by comprising:

detecting the relative position between the screen on which astereoscopic image is displayed and the viewer,

wherein said critical parallax is calculated as a parallax in which thedifference between the inverse of the distance from the point at whichthe optical axes of the right and left eyes intersect to the eyes andthe inverse of the distance from the screen on which the stereoscopicimage is displayed to the eyes has a predetermined value, in the fifthaspect.

Still further, a stereoscopic image displaying method according to aseventh aspect is characterized by the stereoscopic image displayingmethod, characterized in that said critical parallax is corrected on thebasis of the personal characteristics of the viewer, in the fifth orsixth aspect.

Still further, a stereoscopic image displaying method according to aneighth aspect is characterized by the stereoscopic image displayingmethod, characterized by comprising:

calculating the brightness difference, that is, by using one space ofthe divided spaces as a reference, the difference between the brightnessvalue of a stereoscopic image generated in said one space and thebrightness value of a stereoscopic image generated in another spaceadjacent thereto;

moving the other space in parallel with said one space and recalculatingthe brightness difference; and

obtaining the relative position of the other space with respect to saidone space, at which the calculated brightness difference becomesminimal, in any one of the fifth through seventh aspects.

Still further, a computer program according to a ninth aspect ischaracterized by a computer program for generating and displaying astereoscopic image that forms a three-dimensional image on a retina of aviewer, characterized in that said computer is operated to function as:

critical parallax calculating means for calculating a critical parallaxthat is the boundary of a parallax capable of forming athree-dimensional image on a retina of the viewer;

rectangular parallelepiped dimension calculating means for calculatingthe dimensions of a rectangular parallelepiped inscribing a spherehaving a diameter equal to the calculated critical parallax;

space dividing means for dividing a space including an object intomultiple spaces using the calculated rectangular parallelepiped;

gazing point image generating means for generating a stereoscopic imageof the object with respect to a single gazing point for each dividedspace;

gazing point image pasting means for pasting the generated multiplestereoscopic images together to generate a single stereoscopic image;and

image displaying means for displaying the generated single stereoscopicimage.

Still further, a computer program according to a 10th aspect ischaracterized by the computer program, characterized in that saidcomputer is operated:

to function as relative position detecting means for detecting therelative position between the screen on which a stereoscopic image isdisplayed and the viewer, and

to function such that said critical parallax is calculated as a parallaxin which the difference between the inverse of the distance from thepoint at which the optical axes of the right and left eyes intersect tothe eyes and the inverse of the distance from the screen on which thestereoscopic image is displayed to the eyes has a predetermined value,in the ninth aspect.

Still further, a computer program according to an 11th aspect ischaracterized by the computer program, characterized in that saidcomputer is operated to function as correcting means for correcting saidcritical parallax on the basis of the personal characteristics of theviewer, in the ninth or 10th aspect.

Still further, a computer program according to a 12th aspect ischaracterized by the computer program, characterized in that saidcomputer is operated to function as:

means for calculating the brightness difference, that is, by using onespace of the divided spaces as a reference, the difference between thebrightness value of a stereoscopic image generated in said one space andthe brightness value of a stereoscopic image generated in another spaceadjacent thereto;

means for moving the other space in parallel with said one space and forrecalculating the brightness difference; and

means for obtaining the relative position of the other space withrespect to said one space, at which the calculated brightness differencebecomes minimal, in any one of the ninth through 11th aspects.

In the first, fifth and ninth aspects, a critical parallax that is theboundary of the parallax capable of forming a three-dimensional image ona retina of a viewer is calculated, the dimensions of a rectangularparallelepiped within the range of the calculated critical parallax arecalculated, a space including an object is divided into multiple spacesusing the calculated rectangular parallelepiped, a stereoscopic image ofthe object with respect to a single gazing point is generated for eachdivided space including the object, and the generated multiplestereoscopic images are pasted together to generate a singlestereoscopic image and to display and output the single stereoscopicimage. With this configuration, a space is divided using a rectangularparallelepiped within the range of the parallax wherein the viewer doesnot recognize an image as a blurred image, a stereoscopic image isgenerated for each divided space in a way similar to the conventionalmethod, and then these images are pasted together to generate astereoscopic image.

Therefore, the generated stereoscopic image is a stereoscopic imagewithin the range of the critical parallax in any regions thereof, and noblurred three-dimensional image is formed on a retina even if thestereoscopic image is viewed by the viewer from any positions and anyangles.

Hence, all the regions of the stereoscopic image can be visuallyrecognized accurately without using varifocal lenses, and the spatialpositional relationship of the object can be grasped accurately. Inaddition, since image generation can be done simply by pasting only thestereoscopic images each generated in a space having the shape of arectangular parallelepiped in which the object is present. As a result,the load for computer processing can be decreased significantly, and thecost for computation can be reduced.

In the second, sixth and 10th aspects, relative position detecting meansfor detecting the relative position between a screen on which astereoscopic image is displayed and a viewer is provided, and thecritical parallax is calculated as a parallax in which the differencebetween the inverse of the distance from the point at which the opticalaxes of the right and left eyes intersect to the eyes and the inverse ofthe distance from the screen on which the stereoscopic image isdisplayed to the eyes has a predetermined value. Hence, the criticalparallax can be calculated accurately depending on the viewing positionof the viewer, whereby it is possible to display a stereoscopic imageadapted to the viewer.

In the third, seventh and 11th aspects, correcting means for correctingthe critical parallax on the basis of the personal characteristics of aviewer is provided. With this configuration, the calculated criticalparallax is corrected depending on the personal characteristics of theviewer, such as focusing in front of or behind the retina position dueto nearsightedness or farsightedness or the presence or absence of dirton the crystalline lens, whereby the critical parallax can be calculatedmore accurately, and it is possible to display a stereoscopic image moreadapted to the viewer.

In the fourth, eighth and 12th aspects, by using one space of thedivided spaces as a reference, the brightness difference, that is, thedifference between the brightness value of a stereoscopic imagegenerated in this one space and the brightness value of a stereoscopicimage generated in another space adjacent thereto is calculated, theother space is moved in parallel with the one space and the brightnessdifference is recalculated, and the relative position of the other spacewith respect to the one space, at which the calculated brightnessdifference becomes minimal, is obtained.

With this configuration, the gap portion, such as a transparent portionbeing generated at the boundary of the adjacent stereoscopic images whenthe stereoscopic images are simply pasted at their original spacepositions, can be eliminated. Hence, it is possible to generate astereoscopic image that can form a natural three-dimensional image on aretina of the viewer even at the boundary portion.

In the first, fifth and ninth aspects, a space is divided using arectangular parallelepiped within the range of the parallax wherein theviewer does not recognize an image as a blurred image, a stereoscopicimage is generated for each divided space in a way similar to theconventional method, and then these images are pasted together togenerate a stereoscopic image. Therefore, the generated stereoscopicimage is a stereoscopic image within the range of the critical parallaxin any regions thereof, and no blurred three-dimensional image is formedon a retina even if the stereoscopic image is viewed by the viewer fromany positions and any angles.

Hence, all the regions of the stereoscopic image can be visuallyrecognized accurately without using varifocal lenses, and the spatialpositional relationship of the object can be grasped accurately. Inaddition, since image generation can be done simply by pasting only thestereoscopic images each generated in a space having the shape of arectangular parallelepiped in which the object is present. As a result,the load for computer processing can be decreased significantly, and thecost for computation can be reduced.

In the second, sixth and 10th aspects, the critical parallax can becalculated accurately depending on the viewing position of the viewer,whereby it is possible to display a stereoscopic image adapted to theviewer.

In the third, seventh and 11th aspects, the calculated critical parallaxis corrected depending on the personal characteristics of the viewer,such as focusing in front of or behind the retina position due tonearsightedness or farsightedness or the presence or absence of dirt onthe crystalline lens, whereby the critical parallax can be calculatedmore accurately, and it is possible to display a stereoscopic image moreadapted to the viewer.

In the fourth, eighth and 12th aspects, the gap portion, such as atransparent portion being generated at the boundary of the adjacentstereoscopic images when the stereoscopic images are simply pasted attheir original space positions, can be eliminated. Hence, it is possibleto generate a stereoscopic image that can form a naturalthree-dimensional image on a retina of the viewer even at the boundaryportion.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is block diagram showing the configuration of a stereoscopicimage display apparatus according to Embodiment 1 of the presentinvention;

FIG. 2 is a flowchart showing the processing procedure of the CPU of thestereoscopic image display apparatus according to Embodiment 1 of thepresent invention;

FIG. 3 is a view illustrating a critical parallax;

FIG. 4 is a flowchart showing a procedure for calculating the criticalparallax using the CPU of the stereoscopic image display apparatusaccording to Embodiment 1 of the present invention;

FIG. 5 is a view showing a gazing point and a viewpoint that are set atdesired positions in a space to determine the composition of thestereoscopic image of the image data stored in the image data storagesection;

FIG. 6 is a view illustrating a method for obtaining a gazing point of adivided space;

FIG. 7 is a schematic view showing a method for determining a firstdivided space;

FIGS. 8( a) and 8(b) are schematic views showing a method for obtainingthe height and width of a rectangular parallelepiped for the firstdivided space;

FIG. 9 is a view showing a state wherein the first divided space isdisposed;

FIG. 10 is a view showing a state in which rectangular parallelepipedsare disposed as the divided spaces of an object;

FIG. 11 is a view showing examples of stereoscopic images generated inthe first row;

FIG. 12 is a flowchart showing the stereoscopic image pasting processingof the CPU of the stereoscopic image display apparatus according toEmbodiment 1 of the present invention;

FIG. 13 is a schematic view showing a state in which a stereoscopicimage to be adjusted is moved dot by dot;

FIG. 14 is a block diagram showing the configuration of a stereoscopicimage display apparatus according to Embodiment 2 of the presentinvention;

FIG. 15 is a flowchart showing a procedure for calculating the criticalparallax using the CPU of the stereoscopic image display apparatusaccording to Embodiment 2 of the present invention;

FIG. 16 is a schematic view showing the concept of the conventionalstereoscopic image display system;

FIG. 17 is a view showing regions in which image blurring occurs in theconventional stereoscopic image display system; and

FIG. 18 is a view showing regions in which image blurring occurs in thestereoscopic image display system incorporating varifocal lenses.

EXPLANATION OF THE REFERENCE NUMERALS

-   10 Stereoscopic image display apparatus-   11 CPU-   12 Storage means-   13 RAM-   14 Communication means-   15 Input means-   16 Display interface-   17 Auxiliary storage means-   18 Portable recording media-   20 Large screen display unit-   30 Display unit-   40 Viewer sensor-   41 Sensor interface-   121 Image data storage section-   D Critical parallax-   g, g1, g2, . . . , gn Gazing points

DETAILED DESCRIPTION Embodiment 1

A stereoscopic image display apparatus according to Embodiment 1 of thepresent invention will be described below specifically on the basis ofthe drawings. FIG. 1 is a block diagram showing the configuration of thestereoscopic image display apparatus 10 according to Embodiment 1 of thepresent invention.

As shown in FIG. 1, the stereoscopic image display apparatus 10comprises at least a CPU (central processing unit) 11, storage means 12,a RAM 13, communication means 14 for communicating with an externalnetwork, such as Internet, input means 15, a display interface 16 foroutputting display image data to an external large screen display unit20 that can be viewed by multiple viewers, and auxiliary storage means17 that use portable recording media 18, such as DVDs and CDs.

The CPU 11 is connected to the above-mentioned hardware devices of thestereoscopic image display apparatus 10 via an internal bus 19, controlsthe above-mentioned hardware devices, and executes various softwarefunctions according to processing programs stored in the storage means12, such as a program for calculating a critical parallax, a program fordividing a space using a rectangular parallelepiped within the range ofthe critical parallax, a program for generating a stereoscopic image ofan object with respect to a single gazing point, and a program forpasting multiple stereoscopic images together.

The storage means 12 comprises a fixed storage device (hard disk), aROM, etc. being built therein, and stores processing programs obtainedfrom an external computer via the communication means 14 or the portablestorage media 18, such as DVDs and CD-ROMs and required for allowing adisplay unit to function as the stereoscopic image display apparatus 10.The storage means 12 stores not only the processing programs but also,for example, the image data obtained by photographing objects andreceived from an external computer, in the image data storage section121 thereof.

The RAM 13 is formed of a DRAM or the like and stores temporary datagenerated while the software is executed. The communication means 14 isconnected to the internal bus 19. When connected so as to be able tocommunicate with a network, such as Internet or LAN, the communicationmeans 14 transmits and receives data required for the processing.

The input means 15 is a pointing device, such as a mouse, for indicatinga position on the screen or a keyboard for key entering numeric data,such as the horizontal width of the screen on the screen.

The display interface 16 is an LSI board for transmitting display datato the external large screen display unit 20, such as a liquid crystaldisplay device (LCD) or a display device (CRT), for displaying images.

The auxiliary storage means 17 uses the portable recording means 18,such as CDs and DVDS. Programs, data, etc., to be processed by the CPU11, are downloaded to the storage means 12 using the auxiliary storagemeans 17. In addition, data processed by the CPU 11 can be written forbackup using the auxiliary storage means 17.

FIG. 2 is a flowchart showing the processing procedure of the CPU 11 ofthe stereoscopic image display apparatus 10 according to Embodiment 1 ofthe present invention. The CPU 11 of the stereoscopic image displayapparatus 10 first calculates a critical parallax that is the boundaryof a parallax capable of forming a three-dimensional image on a retinaof a viewer (at step S201)

The CPU 11 calculates the dimensions of a rectangular parallelepipedinscribing a sphere having a diameter equal to the calculated criticalparallax (at step S202), reads the image data obtained by photographingan object from the image data storage section 121, and divides a spaceincluding the object into multiple spaces using the calculatedrectangular parallelepiped (at step S203).

The CPU 11 then generates a stereoscopic image of the object withrespect to a single gazing point for each divided space (at step S204).

The CPU 11 sequentially selects each stereoscopic image from among themultiple stereoscopic images (at step S205), and judges whether theobject is included in the selected stereoscopic image (at step S206).When the CPU 11 judges that the object is included in the selectedstereoscopic image (YES at step S206), the CPU 11 stores the selectedstereoscopic image as a component for generating a whole stereoscopicimage in the RAM 13 (at step S207), and judges whether all thestereoscopic images have been selected or not (at step S208). When theCPU 11 judges that the object is not included in the selectedstereoscopic image (NO at step S206), the CPU 11 skips to step S208without storing the stereoscopic image in the RAM 13.

When the CPU 11 judges that all the stereoscopic images have not beenselected (NO at step S208), the CPU 11 returns the processing to stepS205, and the above-mentioned processing is executed repeatedly. Whenthe CPU 11 judges that all the stereoscopic images have been selected(YES at step S208), the CPU 11 pastes the stored multiple stereoscopicimages together (at step S209), and transmits a generated singlestereoscopic image to the external large screen display unit 20 via thedisplay interface 16 (at step S210).

FIG. 3 is a view illustrating a critical parallax. When a viewer sees anobject, the rotation of the eye balls, the thickness adjustment of thecrystalline lenses, etc. are performed so that the optical axes of theright and left eyes 31 and 32 intersect on the object. As a result, thedistance L1 from the point at which the optical axes of the right andleft eyes 31 and 32 intersect to the eyes 31 and 32 is changed, wherebythe relationship between the distance L1 and the distance L2 from thescreen or the like of the display unit 20 to which the object isprojected to the eyes 31 and 32 is also changed.

In other words, when an image of a portion located at a value away fromthe gazing point in the depth direction is viewed, the distance L1 andthe distance L2 are not equal to each other, but have a considerabledifference in distance therebetween. When this difference in distancebecomes larger gradually and reaches a predetermined difference, theviewer recognizes that the image is “blurred.” The parallax in the caseof this difference in distance is defined as critical parallax.

Generally speaking, it is known that an image can be recognized as astereoscopic image without any blurring in the case that the differencebetween the inverse of the distance L1 from the point at which theoptical axes of the right and left eyes 31 and 32 intersect to the eyes31 and 32 and the inverse of the distance L2 from a screen or the liketo which the object is projected to the eyes 31 and 32 is within ±2.Hence, the critical parallax can be calculated according to theprocedure described below.

FIG. 4 is a flowchart showing the detailed processing executed at stepS201 by the CPU 11 of the stereoscopic image display apparatus 10according to Embodiment 1 of the present invention, that is, a procedurefor calculating the critical parallax.

The CPU 11 of the stereoscopic image display apparatus 10 firstcalculates the critical parallax in a real space according to theprocedure described below (at step S401).

Since the difference between the inverse of the distance L1 and theinverse of the distance L2 is within ±2, (Expression 1) is establishedbetween the distance L1 and the distance L2.

(1/L2−2)<1/L1<(1/L2+2)   (Expression 1)

In addition, since the triangle whose base is equal to the parallax dshown in FIG. 3 is similar to the triangle whose base is equal to thedistance E between the two eyes, (Expression 2) is established.

L1=L2×E/(d+E)   (Expression 2)

When (Expression 1) and (Expression 2) are arranged with respect to theparallax d, (Expression 3) can be derived. The absolute value of theparallax d at the boundary determining whether (Expression 3) issatisfied or not is the critical parallax D.

−2×L2×E<d<2×L2×E   (Expression 3)

The CPU 11 calculates the number of pixels of the critical parallax D onthe basis of the calculated critical parallax D, the horizontal width Wof the screen of the large screen display unit 20 and the resolution Rin the horizontal direction of the screen thereof (at step S402). Inother words, the critical parallax D is converted into the number ofpixels Q of the screen displaying the critical parallax D according to(Expression 4).

Q=D×R/W   (Expression 4)

Since the critical parallax D is obtained using the number of pixels ofthe display on the display unit as described above, the viewer does notrecognize that the image of the object is “blurred” inside a spherecentered at the gazing point and having a diameter equal to the criticalparallax D. This range is referred to as a stereoscopic limit.Accordingly, in Embodiment 1, a rectangular parallelepiped inscribingthe stereoscopic limit is calculated, and a space including the objectis divided using the obtained rectangular parallelepiped so that thespace including the object can be divided without leaving any gaps.

FIG. 5 is a view showing a gazing point and a viewpoint that are set atdesired positions inside a space to determine the composition of thestereoscopic image of image data stored in the image data storagesection 121 of the storage means 12. In the example shown in FIG. 5, agazing point g is set near the gravity center of a vehicle serving asthe object, and a viewpoint v is set at a position from which the gazingpoint g is seen.

Furthermore, on the basis of the gazing point g and the viewpoint vreceived, the CPU 11 determines a first rectangular parallelepiped thatis used to divide the space. FIG. 6 is a view illustrating a method forobtaining the gazing point g1 of a divided space. The intersection pointP1 of the line segment connecting the gazing point g to the viewpoint vand the surface of the object represented as a three-dimensional modelis obtained. The point moved from the intersection point P1 toward thegazing point g by a predetermined distance M along the line segment isobtained as the gazing point g1 for a first divided space. The movementdistance M is a distance obtained according to M=L×p (0≦p≦1) wherein acoefficient p is a parameter that is used to adjust an image near theintersection point P1.

Moreover, L is the depth of a rectangular parallelepiped around thegazing point g1 of the first divided space and is calculated on thebasis of the critical parallax D. The shape of the rectangularparallelepiped is adjusted by multiplying the critical parallax D by ashape parameter q (0≦q≦1). In other words, as q is larger, the depthbecomes larger, and the height and width become smaller.

The CPU 11 determines the coordinate of a given point a so that theparallax at the point a on the line segment connecting the viewpoint vto the gazing point g1 of the first divided space does not exceed thevalue obtained by multiplying the critical parallax D by the shapeparameter q. FIG. 7 is a schematic view showing a method for determiningthe first divided space. The CPU 11 determines the coordinate of thepoint a at which the parallax does not exceed the critical parallax D,in the depth direction from the viewpoint v to the gazing point g1 ofthe first divided space. Next, the CPU 11 determines a rectangle D1 sothat the intersection point of the diagonal lines of the rectangle isaligned with the gazing point g1 and so that half the length of thediagonal line is equal to the distance between the gazing point g1 andthe point a.

FIGS. 8( a) and 8(b) are schematic views showing a method for obtainingthe height of a rectangular parallelepiped for the first divided space.As shown in FIG. 8( a), the CPU 11 determines the coordinate of a pointb at which the parallax does not exceed the critical parallax D, on astraight line perpendicular to the plane including the rectangle D1.Next, as shown in FIG. 8( b), the CPU 11 determines a rectangularparallelepiped serving as the first divided space around the gazingpoint g1, the base of which has the shape of the rectangle D1 and theheight of which is twice the distance h between the points a and b.

After determining the depth, height and width of the rectangularparallelepiped being used for space dividing, the CPU 11 disposes therectangular parallelepiped formed around the gazing point g1 as a firstdivided space. FIG. 9 is a view showing a state in which the firstdivided space is disposed. As shown in FIG. 9, one rectangularparallelepiped serves as one divided space, and the center of therectangular parallelepiped is the gazing point g1 of the divided space.Furthermore, the CPU 11 sequentially disposes the rectangularparallelepipeds so that they are adjacent to the rectangularparallelepiped serving as the first divided space in theup-down/right-left/front-rear directions.

FIG. 10 is a view showing a state in which rectangular parallelepipedsare disposed as the divided spaces of an object. As shown in FIG. 10,the CPU 11 sequentially dispose the rectangular parallelepipeds untilthe three-dimensional model showing the object is not included in therectangular parallelepipeds. With this disposition, the parallax of theimage of the three-dimensional model included in each rectangularparallelepiped does not exceed the apex b shown in FIG. 8( a). This isbecause the rectangular parallelepiped to be disposed becomes smaller ina stereoscopic image as it is away from the first rectangularparallelepiped.

When generating a stereoscopic image for each divided space as describedabove, the CPU 11 fixes the viewpoint v and generates stereoscopicimages corresponding to the gazing points g1 , g2, . . . , gn (n: anatural number) of the respective rectangular parallelepipeds serving asthe divided spaces in a way similar to the conventional method.

The stereoscopic images are generated sequentially beginning with adivided space near the viewpoint v. For example, when the rectangularparallelepipeds are disposed in m rows (m: a natural number) in thedepth direction as shown in FIG. 10, the CPU 11 sequentially generatesstereoscopic images beginning with the first row. In the case that thestereoscopic images of the second, third, . . . , kth (k: a naturalnumber, 1≦k≦m) rows are generated, and when divided spaces are concealedcompletely from the viewpoint v by the stereoscopic images generatedearlier, that is, by the stereoscopic images in the first to the (k-1)throws, the generation of the stereoscopic images for the divided spacesis skipped. FIG. 11 is a view showing examples of stereoscopic imagesgenerated in the first row. As described above, stereoscopic images thatcan be seen from the viewpoint v by the viewer are generatedsequentially, and these images are pasted together. Hence, the viewercan recognize the obtained image as a three-dimensional image regardlessof which part of the image displayed on a large screen is viewed by theviewer.

However, if the stereoscopic images are simply pasted together,transparent gap portions are generated at the boundaries of the dividedspaces, and an unnatural three-dimensional image is obtained. To solvethis problem, the CPU 11 moves, from a divided space, another dividedspace adjacent thereto dot by dot so that transparent gap portions areeliminated at the boundaries. FIG. 12 is a flowchart showing thestereoscopic image pasting processing of the CPU 11 of the stereoscopicimage display apparatus 10 according to Embodiment 1 of the presentinvention.

From among the generated multiple stereoscopic images, the CPU 11 of thestereoscopic image display apparatus 10 specifies a stereoscopic imageto be used as a reference for the pasting processing and a stereoscopicimage to be adjusted (at step S1201).

Generally speaking, the stereoscopic image to be used as the referenceis moved sequentially from the stereoscopic image corresponding to therectangular parallelepiped of the first divided space to thestereoscopic image subjected to the pasting processing.

The CPU 11 moves the stereoscopic image to be adjusted toward the gazingpoint of the stereoscopic image to be used as the reference (referencegazing point) by one dot (at step S1202), and judges beforehand whethera point having a brightness value registered as a background color, thatis, a transparent portion through which the background can be seen ispresent or not (at step S1203). FIG. 13 is a schematic view showing astate in which the stereoscopic image to be adjusted is moved dot bydot. In FIG. 13, a stereoscopic image 62 to be adjusted is moved dot bydot toward the viewpoint (reference gazing point) g of a stereoscopicimage 61 to be used as the reference, whereby the background colorportion being present between the stereoscopic image 61 to be used asthe reference and the stereoscopic image 62 to be adjusted, that is, atransparent portion 63, is eliminated.

When the CPU 11 judges that a transparent portion is present (YES atstep S1203), the CPU 11 returns the processing to step S1202 and movesthe stereoscopic image to be adjusted repeatedly dot by dot to carry outapproaching. When the CPU 11 judges that no transparent portion ispresent (NO at step S1203), the CPU 11 calculates the difference betweenthe brightness values at the boundary (at step S1204), and judgeswhether the difference between the brightness values is a predeterminedvalue or less (at step S1205).

When the CPU 11 judges that the difference between the brightness valuesis not the predetermined value or less (NO at step S1205), the CPU 11returns the processing to step S1202, and executes the above-mentionedprocessing repeatedly. When the CPU 11 judges that the differencebetween the brightness values is the predetermined value or less (YES atstep S1205), the CPU 11 judges whether the pasting processing for allthe stereoscopic images has been completed or not (at step S1206).

When the CPU 11 judges that the pasting processing for all thestereoscopic images has been completed (YES at step S1206), the CPU 11transmits a generated stereoscopic image to the large screen displayunit 20 via the display interface 16 (at step S1207). When the CPU 11judges that the pasting processing for all the stereoscopic images hasnot been completed (NO at step S1206), the CPU 11 returns the processingto step S1201, and executes the above-mentioned processing repeatedly.

In the above-mentioned processing, the relative position of thestereoscopic image to be adjusted in the pasting processing with respectto the stereoscopic image to be used as the reference is determineddepending on whether the difference between the brightness values at theboundary is the predetermined value or less.

However, it may be possible that the differences between all thebrightness values in a certain range are calculated and that theposition at which the difference between the brightness values isminimal is used as the relative position of the stereoscopic image to beadjusted with respect to the stereoscopic image to be used as thereference. With this configuration, the gap portion, such as atransparent portion being generated at the boundary of the adjacentstereoscopic images when the stereoscopic images are simply pasted attheir original space positions, can be eliminated more effectively.

Hence, it is possible to generate a stereoscopic image that can form anatural three-dimensional image on a retina of the viewer even at theboundary portion.

As described above, in Embodiment 1, a space is divided using arectangular parallelepiped within the range of the parallax wherein theviewer does not recognize an image as a blurred image, a stereoscopicimage is generated for each divided space in a way similar to theconventional method, and then these images are pasted together togenerate a stereoscopic image. Hence, the generated stereoscopic imageis a stereoscopic image within the range of the critical parallax in anyregions thereof, and no blurred three-dimensional image is formed on aretina even if the stereoscopic image is viewed by the viewer from anypositions and any angles.

Hence, all the regions of the stereoscopic image can be visuallyrecognized accurately without using varifocal lenses, and the spatialpositional relationship of the object can be grasped accurately. Inaddition, since image generation can be done simply by pasting only thestereoscopic images each generated in a space having the shape of arectangular parallelepiped in which the object is present. As a result,the load for computer processing can be decreased significantly, and thecost for computation can be reduced.

Embodiment 2

A stereoscopic image display apparatus according to Embodiment 2 of thepresent invention will be described below specifically on the basis ofthe drawings. FIG. 14 is a block diagram showing the configuration ofthe stereoscopic image display apparatus 10 according to Embodiment 2 ofthe present invention. As shown in FIG. 14, since the configuration ofthe stereoscopic image display apparatus 10 according to Embodiment 2 ofthe present invention is similar to that according to Embodiment 1,similar components are designated by the same numerals, and theirdetailed descriptions are omitted. Embodiment 2 is characterized in thata stereoscopic image is transmitted to an external display unit 30 thatcan be seen by a single viewer instead of the large screen display unit20 that can be seen by numerous viewers, and that the critical parallaxbeing used as the reference for generating the stereoscopic image iscorrected on the basis of the distance to the screen from the viewerdetected using a viewer sensor 40, the distance between the eyes of theviewer, etc.

The input means 15 is a pointing device, such as a mouse, for indicatinga position on the screen or a keyboard for key entering numeric data,such as the horizontal width of the screen on the screen.

The display interface 16 is an LSI board for transmitting display datato the external display unit 30, such as a liquid crystal display device(LCD) or a display device (CRT), for displaying images.

The sensor interface 41 transmits the signal detected using the viewersensor 40 disposed externally to the CPU 11. The sensor interface 41 is,for example, an optical sensor, a supersonic sensor or the likeinstalled above the screen of the display unit 30, and the CPU 11calculates the distance to the viewer, the distance between the eyes ofthe viewer, etc. on the basis of the obtained detection signal.

Generally speaking, it is known that an image can be recognized as astereoscopic image without any blurring in the case that the differencebetween the inverse of the distance L1 from the point at which theoptical axes of the right and left eyes 31 and 32 intersect to the eyes31 and 32 and the inverse of the distance L2 from a screen or the liketo which the object is projected to the eyes 31 and 32 is within ±2.However, it is assumed that the value of the difference may varydepending on personal characteristics, such as differences in thedistance of the viewer to the display screen and in the distance betweenthe eyes of the viewer

Hence, in Embodiment 2, to absorb the variation in the above-mentionedallowable limit, the CPU 11 calculates the allowable difference Zbetween the inverse of the distance L1 and the inverse of the distanceL2 using the distance to the viewer, the distance between the eyes ofthe viewer, etc. calculated on the basis of the signal detected usingthe viewer sensor 40.

FIG. 15 is a flowchart showing a procedure for calculating a criticalparallax using the CPU 11 of the stereoscopic image display apparatus 10according to Embodiment 2 of the present invention. The CPU 11 of thestereoscopic image display apparatus 10 obtains the detection signaldetected using the viewer sensor 40 (at step S1501), and calculates thedistance L2 from the screen or the like of the display unit 20 to whichan object is projected to the eyes 31 and 32, the distance E between theeyes of the viewer, etc. (at step 1502).

The CPU 11 calculates the allowable difference Z between the inverse ofthe distance L1 and the inverse of the distance L2 (at step 1503), andcalculates the critical parallax in a real space according to theprocedure described below (at step 1504). Since the difference betweenthe inverse of the distance L1 and the inverse of the distance L2 iswithin ±Z, (Expression 5) is established between the distance L1 and thedistance L2.

(1/L2−Z)<1/L1<(1/L2+Z)   (Expression 5)

In addition, since the triangle whose base is equal to the parallax dshown in FIG. 3 is similar to the triangle whose base is equal to thedistance E between the two eyes, (Expression 6) is established.

L1=L2×E/(d+E)   (Expression 6)

When (Expression 5) and (Expression 6) are arranged with respect to theparallax d, (Expression 7) can be derived. The absolute value of theparallax d at the boundary determining whether (Expression 7) issatisfied or not is the critical parallax D.

−Z×L2×E<d<Z×L2×E   (Expression 7)

However, there are differences among individuals in the range in whichimage blurring occurs depending on the personal characteristics of theviewer, such as focusing in front of or behind the retina position dueto nearsightedness or farsightedness or the presence or absence of dirton the crystalline lens. A coefficient for correcting the criticalparallax D so as to be more adapted to the personal characteristics ofthe viewer is set via the input means 15.

As a result, it is possible to generate a stereoscopic image moreadapted to the viewer.

The CPU 11 calculates the number of pixels of the critical parallax D onthe basis of the calculated critical parallax D, the horizontal width Wof the screen of the display unit 30 and the resolution R in thehorizontal direction of the screen thereof (at step S1505). In otherwords, the critical parallax D is converted into the number of pixels Qof the screen displaying the critical parallax D according to(Expression 8).

Q=D×R/W   (Expression 8)

Since the critical parallax D is obtained as the number of pixels of thedisplay on the display unit as described above, the viewer does notrecognize that the image of the object is “blurred” in a sphere centeredat the gazing point and having a diameter equal to the critical parallaxD. This range is referred to as a stereoscopic limit. Accordingly, inEmbodiment 1, a rectangular parallelepiped inscribing the stereoscopiclimit is calculated, and a space including the object is divided usingthe obtained rectangular parallelepiped so that the space including theobject can be divided without leaving any gaps.

Then, as in Embodiment 1, a stereoscopic image is generated for eachdivided space, and the generated multiple stereoscopic images are pastedtogether to generate a single stereoscopic image.

Hence, it is possible to view an accurate stereoscopic image regardlessof, for example, the viewing posture, the viewing direction, etc. of theviewer.

According to Embodiment 2 described above, the critical parallax can becalculated accurately depending on the viewing position of the viewer,whereby it is possible to display a stereoscopic image adapted to theviewer. Furthermore, the calculated critical parallax is correcteddepending on the personal characteristics of the viewer, such asfocusing in front of or behind the retina position due tonearsightedness or farsightedness or the presence or absence of dirt onthe crystalline lens, whereby the critical parallax can be calculatedmore accurately, and it is possible to display a stereoscopic image moreadapted to the viewer.

As this description may be embodied in several forms without departingfrom the spirit of essential characteristics thereof, the presentembodiments are therefore illustrative and not restrictive, since thescope is defined by the appended claims rather than by descriptionpreceding them, and all changes that fall within metes and bounds of theclaims, or equivalence of such metes and bounds thereof are thereforeintended to be embraced by the claims.

1. A stereoscopic image display apparatus for generating and displayinga stereoscopic image that forms a three-dimensional image on a retina ofa viewer, comprising: a critical parallax calculating unit forcalculating a critical parallax that is the boundary of a parallaxcapable of forming a three-dimensional image on a retina of the viewer;a rectangular parallelepiped dimension calculating unit for calculatingthe dimensions of a rectangular parallelepiped inscribing a spherehaving a diameter equal to the calculated critical parallax; a spacedividing unit for dividing a space including an object into multiplespaces using the calculated rectangular parallelepiped; a gazing pointimage generating unit for generating a stereoscopic image of the objectwith respect to a single gazing point for each divided space; a gazingpoint image pasting unit for pasting the generated multiple stereoscopicimages together to generate a single stereoscopic image; and an imagedisplaying unit for displaying the generated single stereoscopic image.2. The stereoscopic image display apparatus as set forth in claim 1,further comprising a relative position detecting unit for detecting therelative position between the screen on which a stereoscopic image isdisplayed and the viewer, wherein said critical parallax is calculatedas a parallax in which the difference between the inverse of thedistance from the point at which the optical axes of the right and lefteyes intersect to the eyes and the inverse of the distance from thescreen on which the stereoscopic image is displayed to the eyes has apredetermined value.
 3. The stereoscopic image display apparatus as setforth in claim 1, wherein said critical parallax calculating unit isequipped with a correcting unit for correcting said critical parallax onthe basis of the personal characteristics of the viewer.
 4. Thestereoscopic image display apparatus as set forth in claims 1, whereinsaid gazing point image pasting unit comprises: a unit for calculatingthe brightness difference, that is, by using one space of the dividedspaces as a reference, the difference between the brightness value of astereoscopic image generated in said one space and the brightness valueof a stereoscopic image generated in another space adjacent thereto; aunit for moving the other space in parallel with said one space and forrecalculating the brightness difference; and a unit for obtaining therelative position of the other space with respect to said one space, atwhich the calculated brightness difference becomes minimal.
 5. Astereoscopic image displaying method for generating and displaying astereoscopic image that forms a three-dimensional image on a retina of aviewer, comprising: calculating a critical parallax that is the boundaryof a parallax capable of forming a three-dimensional image on a retinaof the viewer; calculating the dimensions of a rectangularparallelepiped inscribing a sphere having a diameter equal to thecalculated critical parallax; dividing a space including an object intomultiple spaces using the calculated rectangular parallelepiped;generating a stereoscopic image of the object with respect to a singlegazing point for each divided space; pasting the generated multiplestereoscopic images together to generate a single stereoscopic image;and displaying the generated single stereoscopic image.
 6. Thestereoscopic image displaying method as set forth in claim 5,comprising: detecting the relative position between the screen on whicha stereoscopic image is displayed and the viewer, wherein said criticalparallax is calculated as a parallax in which the difference between theinverse of the distance from the point at which the optical axes of theright and left eyes intersect to the eyes and the inverse of thedistance from the screen on which the stereoscopic image is displayed tothe eyes has a predetermined value.
 7. The stereoscopic image displayingmethod as set forth in claim 5, wherein said critical parallax iscorrected on the basis of the personal characteristics of the viewer. 8.The stereoscopic image displaying method as set forth in claim 5,comprising: calculating the brightness difference, that is, by using onespace of the divided spaces as a reference, the difference between thebrightness value of a stereoscopic image generated in said one space andthe brightness value of a stereoscopic image generated in another spaceadjacent thereto; moving the other space in parallel with said one spaceand recalculating the brightness difference; and obtaining the relativeposition of the other space with respect to said one space, at which thecalculated brightness difference becomes minimal.
 9. A computer programproduct stored on a computer readable medium for controlling a computerthat generates and displays a stereoscopic image that forms athree-dimensional image on a retina of a viewer, that said computerbeing operated to function as: critical parallax calculating means forcalculating a critical parallax that is the boundary of a parallaxcapable of forming a three-dimensional image on a retina of the viewer;rectangular parallelepiped dimension calculating means for calculatingthe dimensions of a rectangular parallelepiped inscribing a spherehaving a diameter equal to the calculated critical parallax; spacedividing means for dividing a space including an object into multiplespaces using the calculated rectangular parallelepiped; gazing pointimage generating means for generating a stereoscopic image of the objectwith respect to a single gazing point for each divided space; gazingpoint image pasting means for pasting the generated multiplestereoscopic images together to generate a single stereoscopic image;and image displaying means for displaying the generated singlestereoscopic image.
 10. The computer program product as set forth inclaim 9, wherein said computer is operated: to function as relativeposition detecting means for detecting the relative position between thescreen on which a stereoscopic image is displayed and the viewer, and tofunction such that said critical parallax is calculated as a parallax inwhich the difference between the inverse of the distance from the pointat which the optical axes of the right and left eyes intersect to theeyes and the inverse of the distance from the screen on which thestereoscopic image is displayed to the eyes has a predetermined value.11. The computer program product as set forth in claim 9, wherein saidcomputer is operated to function as correcting means for correcting saidcritical parallax on the basis of the personal characteristics of theviewer.
 12. The computer program product as set forth in claim 9,wherein said computer is operated to function as: means for calculatingthe brightness difference, that is, by using one space of the dividedspaces as a reference, the difference between the brightness value of astereoscopic image generated in said one space and the brightness valueof a stereoscopic image generated in another space adjacent thereto;means for moving the other space in parallel with said one space and forrecalculating the brightness difference; and means for obtaining therelative position of the other space with respect to said one space, atwhich the calculated brightness difference becomes minimal.